A bias correction for the minimum error rate in cross-validation

Tuning parameters in supervised learning problems are often estimated by cross-validation. The minimum value of the cross-validation error can be biased downward as an estimate of the test error at that same value of the tuning parameter. We propose a simple method for the estimation of this bias that uses information from the cross-validation process. As a result, it requires essentially no additional computation. We apply our bias estimate to a number of popular classifiers in various settings, and examine its performance.Comment: Published in at http://dx.doi.org/10.1214/08-AOAS224 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Exact Post-Selection Inference for Sequential Regression Procedures

We propose new inference tools for forward stepwise regression, least angle regression, and the lasso. Assuming a Gaussian model for the observation vector y, we first describe a general scheme to perform valid inference after any selection event that can be characterized as y falling into a polyhedral set. This framework allows us to derive conditional (post-selection) hypothesis tests at any step of forward stepwise or least angle regression, or any step along the lasso regularization path, because, as it turns out, selection events for these procedures can be expressed as polyhedral constraints on y. The p-values associated with these tests are exactly uniform under the null distribution, in finite samples, yielding exact type I error control. The tests can also be inverted to produce confidence intervals for appropriate underlying regression parameters. The R package "selectiveInference", freely available on the CRAN repository, implements the new inference tools described in this paper.Comment: 26 pages, 5 figure

The Exclusive View v. The Non-Exclusive View: Can a Creditor’s Claim be Dismissed for Failing to Provide Supporting Documentation?

(Excerpt) May a creditor’s claim be dismissed simply because he failed to provide supporting documentation in violation of Federal Rule of Bankruptcy Procedure 3001? The answer depends on which jurisdiction the creditor is pursuing its claim in. Courts are currently sharply divided on the issue. If the creditor is fortunate enough to be in a jurisdiction which follows the “exclusive” view, which is the majority rule, the answer to this problem will be yes. However, if the creditor happens to be in a jurisdiction which follows the “non-exclusive” view, which is the minority rule, the answer to this problem will be no. The allowance of a creditor’s proof of claim is governed by 11 U.S.C. § 502. Section 502 provides that a creditor’s proof of claim should be allowed except if the claim falls into one of the nine categories listed in section 502(b). According to the “exclusive” view, a creditor’s claim can only be dismissed for one of the nine reasons listed in section 502(b). Although failure to attach supporting documentation to a proof of claim is a violation of Rule 3001, courts that follow the “exclusive” view do not dismiss a creditor’s proof of claim for this violation because this violation is not one of those nine reasons listed in section 502(b). By contrast, the “nonexclusive” view does allow a creditor’s claim to be dismissed for reasons other than those specifically stated in section 502(b). Therefore, because failure to attach supporting documentation to a proof of claim is a violation of Rule 3001, courts which follow the “non-exclusive” view will dismiss a creditor’s proof of claim even though noncompliance with the Rule is not one of those nine reasons listed in section 502(b). This article discusses the difference between the “exclusive” view and the “nonexclusive” view. First, this article examines the provisions of federal law at issue in this court split: 11 U.S.C. § 502 and Federal Rule of Bankruptcy Procedure 3001. Second, this article explains the differences between the “exclusive” view and the “nonexclusive” view, principally by analyzing two recent cases: B-Line v. Kirkland (In re Kirkland), 379 B.R. 341 (B.A.P. 10th Cir. 2007), which exemplifies the “exclusive view,” and In re Tran, 369 B.R. 312 (S.D. Tex. 2007), which exemplifies the “nonexclusive view.” Finally, this article concludes by discussing some important implications of this issue, specifically, the implications these two different views have on large third party debt buyers. These third party debt buyers, such as B-Line and eCast, buy thousands of unsecured claims from credit card companies for cents on the dollar and then seek to collect these claims from debtors in bankruptcy. Depending on which of the two views is applied to their often unsupported proofs of claim these companies stand to gain, or lose, millions of dollars